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Abstract — Statistical  Mechanics  has  proven  to  be  a  useful 
model  for  drawing  inferences  about  the  collective  behavior  of 
individual  objects  that  interact  according  to  a  known  force  law 
(which  for  a  more  general  usage  is  referred  to  as  interacting 
units.).  Collective  behavior  is  determined  not  by  computing 
F  =  ma  for  each  interacting  unit  because  the  problem  is 
mathematically  intractable.  Instead,  one  computes  the  partition 
function  for  the  collection  of  interacting  units  and  predicts  statis¬ 
tical  behavior  from  the  partition  function.  Statistical  mechanics 
was  unified  with  Bayesian  inference  by  Jaynes  who  demonstrated 
that  the  partition  function  assignment  of  probabilities  via  the 
interaction  Hamiltonian  is  the  solution  to  a  Bayesian  assignment 
of  probabilities  based  on  the  maximum  entropy  method  with 
known  means  and  standard  deviations.  Once  this  technique  has 
been  applied  to  a  variety  of  problems  and  obtained  a  solution, 
one  can,  of  course,  solve  the  inverse  problem  to  determine  what 
interaction  model  gives  rise  to  a  given  probability  assignment. 
Probabilistic  networks  are  important  modeling  tools  in  a  variety 
of  applications  including  social  networks.  We  explore  the  usage  of 
statistical  mechanics  as  a  mechanism  to  solve  the  inverse  problem 
to  determine  the  underlying  interaction  model  that  gives  rise  to 
the  probabilistic  network 
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I.  Introduction 

Most  academic  discussions  of  distributed  sensor  networks 
focus  on  the  problems  associated  with  the  fusion  of  data 
or  the  fusion  of  information.  While  these  issues  are  impor¬ 
tant,  they  are  not  the  only  real  issue  along  the  path  that 
motivates  the  reasons  for  distribution  of  sensors.  Networking 
of  the  sensors  enables  the  sharing  of  data  from  similar  as 
well  as  disparate  sensors,  the  fusing  of  the  sensor  data  into 
information,  and  finally  the  synthesis  of  the  information  into 
knowledge.  Information  is  power  is  a  useful  maxim  only  if  it 
leads  to  better  decisions  which  in  turn  result  in  better  actions 
by  the  decisions  maker.  Sensor  networks  are  creating  tsunami 
of  data  that  overwhelm  but  do  not  inform  the  end  user  of 
networks.  With  that  in  mind,  we  need  to  ask  what  are  the 
methodologies  that  can  dampen  out  the  tsunami  of  data  to 
achieve  effective  knowledge  that  leads  to  viable  actions  by 
the  end  user?  While  issues  such  as  timeliness  and  ordering  of 
the  data,  fusion  methodologies  for  the  data,  reliability  of  the 
network,  information  flow  within  the  network,  delineating  the 
difference  between  data  and  usable  information  are  important; 
knowledge  synthesis  from  the  information  provided  by  the 
network  is  the  ultimate  goal. 

Distributed  sensor  networks  are  a  topic  of  interest  to  both 
the  military  and  civilians  because  they  offer  the  possibility 
of  increasing  domain  awareness  in  a  given  region.  Complete 
situational  awareness  can  occur  only  when  sensors  that  op¬ 
erate  at  different  frequencies  and  with  different  operational 
characteristics  are  effectively  combined  to  provide  a  near-real 
time  understanding  of  what  is  happening  within  a  region.  The 
synthesized  "picture"  can  then  be  combined  with  information 
from  other  regions/domains  to  synthesize  information  that  is  of 
the  type  that  is  actionable-  accurate  and  correct  decisions  can 
be  formed  based  on  the  knowledge  that  has  been  synthesized 
from  the  distributed  sensor  networks.  From  the  perspective 
of  knowledge,  however,  the  path  to  the  future  for  networks 
that  lead  to  knowledge  synthesis  is  incomplete  without  the 
consideration  of  the  semantic  aspect  of  information —  taking 
individual  observations  and  judgments  and  making  them  part 
of  the  fusion  problem.  Ultimately,  this  fusion  problem  has  to 
be  considered  from  a  perspective  of  an  end  user...  someone 
whose  career  and  life  is  on  the  line  based  on  a  decision  to  use 
or  not  use  the  information  for  certain  ends. 


1109 


Report  Documentation  Page 


Form  Approved 
OMB  No.  0704-0188 


Public  reporting  burden  for  the  collection  of  information  is  estimated  to  average  1  hour  per  response,  including  the  time  for  reviewing  instructions,  searching  existing  data  sources,  gathering  and 
maintaining  the  data  needed,  and  completing  and  reviewing  the  collection  of  information.  Send  comments  regarding  this  burden  estimate  or  any  other  aspect  of  this  collection  of  information, 
including  suggestions  for  reducing  this  burden,  to  Washington  Headquarters  Services,  Directorate  for  Information  Operations  and  Reports,  1215  Jefferson  Davis  Highway,  Suite  1204,  Arlington 
VA  22202-4302.  Respondents  should  be  aware  that  notwithstanding  any  other  provision  of  law,  no  person  shall  be  subject  to  a  penalty  for  failing  to  comply  with  a  collection  of  information  if  it 
does  not  display  a  currently  valid  OMB  control  number. 


1.  REPORT  DATE 

JUL  2008 

2.  REPORT  TYPE 

3.  DATES  COVERED 

00-00-2008  to  00-00-2008 

4.  TITLE  AND  SUBTITLE 

5a.  CONTRACT  NUMBER 

Design  as  a  Fusion  Problem 

5b.  GRANT  NUMBER 

5c.  PROGRAM  ELEMENT  NUMBER 

6.  AUTHOR(S) 

5d.  PROJECT  NUMBER 

5e.  TASK  NUMBER 

5f.  WORK  UNIT  NUMBER 

7.  PERFORMING  ORGANIZATION  NAME(S)  AND  ADDRESS(ES) 

Naval  Surface  Warfare  Center  Dahlgren,Code  Q-23  Electromagnetic  and 
Sensor  Systems  Department, 18444  Frontage  Road  Suite 

328, Dahlgren,VA, 22448-5161 

8.  PERFORMING  ORGANIZATION 

REPORT  NUMBER 

9.  SPONSORING/MONITORING  AGENCY  NAME(S)  AND  ADDRESS(ES) 

10.  SPONSOR/MONITOR'S  ACRONYM(S) 

1 1 .  SPONSOR/MONITOR' S  REPORT 
NUMBER(S) 

12.  DISTRIBUTION/AVAILABILITY  STATEMENT 

Approved  for  public  release;  distribution  unlimited 

13.  SUPPLEMENTARY  NOTES 

11th  International  Conference  on  Information  Fusion,  June  30  ?  July  3,  2008,  Cologne,  Germany. 

14.  ABSTRACT 

see  report 

15.  SUBJECT  TERMS 

1 

1 

16.  SECURITY  CLASSIFICATION  OF: 


a.  REPORT 

unclassified 


b.  ABSTRACT 

unclassified 


c.  THIS  PAGE 

unclassified 


17.  LIMITATION  OF 
ABSTRACT 

Same  as 
Report  (SAR) 


18.  NUMBER 
OF  PAGES 

8 


19a.  NAME  OF 
RESPONSIBLE  PERSON 


Standard  Form  298  (Rev.  8-98) 

Prescribed  by  ANSI  Std  Z39-18 


From  this  perspective,  decisions  are  more  like  designs, 
weaved  from  a  tapestry  of  colors  and  patterns  which  are 
contradictory.  When  we  think  of  a  decision  like  a  picture,  the 
goals  of  the  fusion  problem  become  quite  different,  and  almost 
unrecognizable.  So  how  do  we  design  a  fusion  problem?  We 
don’t  know  how  to  codify  Design  as  a  specific  law,  but  we 
recognize  it  when  we  see  it.  A  design  is  created  from  disparate 
elements  that  are  combined  to  form  a  composition  (true  of 
artistic  or  engineering  design).  Each  discipline  has  "atoms  of 
composition"  or  atomic  units  that  are  fused  together  to  form  a 
specific  instance  of  a  design.  In  artistic  designs,  the  method  of 
fusion  is  based  on  the  individual  interpretations  of  the  group 
aesthetic  that  falls  under  the  aegis  of  "artistic  temperament". 
For  engineers,  the  group  aesthetic  must  have  some  familiarity 
with  the  requirements  of  "physical  reality".  It  is  difficult  to 
arrive  at  a  mathematical  criteria  that  helps  clarify  the  direct 
problem  of  the  process  of  composition. 

"Probability  is  the  logic  of  reasoning  in  the  presence  of 
uncertainty."  Assignment  of  probabilities  based  on  available 
information  is  Bayesian  reasoning.  This  provides  a  solution  of 
assignment  of  probabilities  as  a  Maximum  Entropy  Problem 
(MEP).  Thus  a  decomposition  of  a  design  into  probabilities 
can  be  viewed  as  a  solution  to  a  MEP.  Given  that  MEP  is 
the  method  that  provides  the  answer,  then  we  can  ask  the 
question:  "What  is  the  Maximum  Entropy  problem  that  an 
assignment  of  design  weights  of  the  composition  an  answer 
to?"  Being  able  to  answer  this  question  for  "design  classes" 
provides  us  a  means  to  classify  Design  as  a  mathematical 
problem.  Design  algorithms  can  then  be  ordered  in  terms 
of  the  complexity  of  the  problem  solved.  By  providing  the 
mathematical  basis  for  synthesizing  design  as  a  composition, 
we  hope  to  provide  a  pathway  to  analyze  decisions  from 
the  perspective  of  design.  This  would  allow  decisions  to  be 
understood  on  a  more  quantative  basis  and  ultimately  bring 
the  decision  maker  in  the  fusion  process  in  a  more  analytical 
process. 

II.  Probability  Issues  Arising  in  Connection 
Models 

A.  Maximum  Entropy  Procedure 

The  expected  value  of  the  "surprise  value"  of  a  probability 
vector  |p)  is 

n 

H{n )  =  —  (p|  In p)  =  —  lnpj.  (surprise  value) 

i-  i 

(See  Papoulis  for  the  details  of  the  background  in  this  section 
[9].)  Note  that  for  a  deterministic  probability  vector,  the 
entropy  is  zero,  while  for  a  uniform  vector,  the  entropy  is 
ln(n).  The  degree  of  uncertainty  (which  is  equivalent  to  the 
surprise  value)  in  the  information  is  defined  as 

n 

S[|p)  ;n]  =  -k  (p\  In p)  =  -k^pj  In pp,  (Entropy) 

2=0 

Most  applications  do  not  permit  one  to  measure  the  probabil¬ 
ities  Pi  associated  with  a  physical  variable  f{xf),  instead  the 


expected  value  (f(x))  is  measured.  Probabilities  are  connected 
to  expected  values  by  noting  the  formula 

n 

E  [f(x)]  =  ( p\f(x ))  =  (Expected  Value) 

i-  0 

Maximum  Entropy  Method  can  be  sumarized  as:  The  goal 
is  to  find  an  assignment  to  the  probability  vector  |p)  subject 
to  available  information.  Problem  is  to  find  an  assignment 
of  probabilities  that  maximizes  the  entropy.  For  example,  we 
always  know  |p)  is  normalized 

n 

<1  |p)  =  ^^Pi  =  1,  (Condition  1) 

*= o 

There  is  a  known  assignment  of  the  means,  ar  (  r  = 
1,2,3 ...m)  which  satisfy: 

n 

{ P\9r  O)))  =  E  [gr( |x))]  =  ’Y^Pigr{xi)  =  ar. 

2=1 

(Condition  2) 

The  Method  of  Lagrange  Undetermined  Multipliers  is  a  stan¬ 
dard  method  for  solving  an  optimization  problem  subject  to 
constraints.  The  function  we  want  to  maximize  is  the  entropy, 
so  it  is  adjoined  to  the  free  parameters  (Lagrange  undetermined 
multipliers)  times  the  equations  for  the  constraints  to  form  a 
Lagrangian.  These  two  condition  the  principle  "The  distribu¬ 
tion,  |p) ,  that  maximizes  the  uncertainty  in  the  expected  value 
subject  to  the  constraint  of  the  available  information ”  provides 
such  an  assignment  that  gives  the  probability  assignment. 

Example  (Assignment  of  probabilities  with  no  known 
information):  The  Lagrangian  for  entropy  maximization  is 

L  =  -(p\  lnp)  -  A0  ((l|p)  -  1) 


Thus  the  least  informative  assignment  of  the  probabilities 
when  only  the  normalization  condition  is  known  is  the  as¬ 
signment  that  the  probabilities  are  uniformly  distributed: 

Pi  =  (p|e»)  =  -•  (1) 

n 

This  provides  an  explanation  for  Occam’s  Razor:  "All  things 
being  equal,  the  simplest  explanation  is  the  best",  e.g.  uniform 
likelihood  of  assignment  of  truth  to  all  explanations  when 
wer  are  in  a  state  of  complete  ignorance.  Einstein’s  Razor: 
"Keep  it  as  simple  as  possible-but  not  simpler."  This  implies 
that  additional  knowledge  beyond  the  means  is  necessary  to 
determine  the  probability  assignment  for  propositions. 

Example:  The  Lagrangian  for  entropy  maximization  (take 
k  =  1)  with  known  mean,  ar.  is  given  by 


L  =  -  (p|  lnp)  -  A0  (<l|p>  -  1)  (2) 

m 

[(P\9r  (\x)))  -  Otr] 

r—1 

1)  To  minimize  with  respect  to  the  probabilities,  we  com¬ 
pute  the  particular  state  (p|ej)  such  that  d®J\ey  is  a 
minimum: 


dL 

9  (p|e») 


m 

=  0  =  In pj  +  1  +  Ao  +  A rgr  (xfj ,  (3) 

r—1 
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2)  Solving  for  the  state’s  probability,  (p|e*)  =  Pi,  gives 

(p\ei)  =  e-(^+i+Er=i  *-»-(**)) 

—  g[ — (^O  +  l  +  ^Isl^i)))] 

(probability  assignment) 


3)  Substituting  probabilities  into  Condition  1  gives  first 
Lagrange  undetermined  multiplier  as 

n 

e(A0+i)  _  e(-  E^Li  Kgr(xi)) 

i= 1 
n 

=  ^V<A|s(Xi)).  (4) 

i— 1 

4)  Other  Lagrange  undetermined  multipliers  are  obtained 
by  substituting  the  probability  assignment  into  Condition 
2 


(P\9r  (|*)))  =Yje-W9(Xi))gr{xi)  =  a,e(Ao+1) 
i=l 

(5) 

which  solves  the  problem  of  assigning  the  probabilities. 

5)  By  defining  the  partition  function  as  Z 

e(A0+i)  _  ^7  (partition  function) 

6)  the  Hamiltonian  H  is  defined  as 


m 


y:  Kgr  (xi) , 

r_i 

(6) 

(Mg(xi))  ■ 

(7) 

7)  The  probability  assignment  becomes 

exp(-JTi) 

(P  |e*>  =  Pi  = - ^ - ,  (8) 

which  is  a  familiar  expression  to  physicists. 

Thus,  knowledge  of  the  variance  leads  to  be  an  exponen¬ 
tial  assignment  to  the  probabilities.  The  physical  connection 
suggested  this  solution  allows  one  to  connect  probabilty  as¬ 
signments  to  physical  interaction  models  as  we  now  show. 


B.  Maximum  Entropy  Applied  to  Connection  Models 

A  connection  model  is  a  technique  to  that  allows  us  to 
breakdown  or  create  an  infrastructure  in  terms  of  information 
sources/sinks  and  links  which  may  or  may  not  "share"  the 
information.  Thus  we  tend  to  think  of  the  mathematical 
structure  in  graph  theoretical  terms.  An  alternative  way  to 
think  about  this  is  in  physical  terms  where  the  links  represent 
interactions,  while  the  nodes  are  sources  for  interactions  such 
as  a  particle  "interacting"  with  another  particle. 

Note,  these  notions  are  adopted  from  Harary  [4].  There 
exists  a  connection  from  the  source  (particle)  i  to  j  if 
there  is  a  sequence  of  sources  i,  i\,  12, ...,  ik,  j  such  that 
(i,  i\),  (zi,Z2) , ...,  (ik,  j)  that  have  some  type  of  connection. 
If  there  are  connections  between  each  of  the  (™)  distinct 
pairs  of  nodes,  then  the  connection  model  is  said  to  be 


totally  connected.  An  adjacency  matrix  ,  A,  for  an  m— edge 
connection  model  is  the  matrix 

all  a12  •  •  •  O-lm-1  alm 


A  = 


0  110  0 
10  1-1 
11... 
o  .  .  .  . 

0  1  •  •  0 


(9) 


®ml  &  m2  '  '  ‘  Q>rnm  —  1  Q>rnm 


where  a,j  =  1  if  source  i  is  connected  to  sourse  j  and  zero  if 
it  isn’t.  A  source  is  not  connected  to  itself  (no  self-interaction), 
so  an  =  0.  An  adjacency  matrix  is  symmetric,  so  A=  A1. 

A  random  connection  model  can  be  defined  in  a  variety  of 
fashions  with  the  randomness  used  to  condition  some  aspect 
of  the  connection  model.  Assume  a  fixed  set  of  vertices  with 
a  rule  for  assigning  connection  to  the  sources.  A  connection 
model  with  a  source  set  G  =  {1,  2, ...,  n}  and  edge  connection 
state  E  =  {i  :  X(i),i  =  l,2...,n}  where  X (i)  are  indepen¬ 
dent  random  variables  with  a  probability  rule  for  assignment 
of  a  connection  for  ( j  =  1,  2, ...,  n ): 


p{x(i)  =  j}  =  f(j) 


(10) 


The  simplest  rule  for  a  connection  is  f(j)  =  const.,  which 
is  a  uniform  distribution  or  equal  probability  assignment  of 
a  connection  between  sources.  For  each  particle  i ,  randomly 
select  a  connection  j  and  randomly  decide  to  connect  a  particle 
to  another  particle  from  the  source  set.  A  connection  model 
constructed  in  such  a  manner  is  referred  to  as  a  random 
connection  model. 

Example:  For  an  arbitrary  connection  model  that  has  a 
weight,  Wij,  associated  with  each  connection  between  source 
(i,j),  we  can  associate  a  number  that  can  be  defined  as  a 
probability  by  the  assignment: 


p  _[  if  (*,j)  is  connected 

|  0  if  (i,j)  is  not  connected. 


(11) 


Definition:  One  can  define  the  Gibbs  entropy  of  an  en¬ 
semble,  £,  of  connection  models  as 

S£  =  ~YJP(G)HP(G)),  (12) 

Ges 

subject  to  the  normalization  requirement  that  the  probability  of 
instances  of  the  connection  model  that  occur  in  an  ensemble 
£  as 

Ep(G)  =  L  (13) 

Ge£ 

Example:  Using  MEP  formalism  with  known  prior  infor¬ 
mation  about  the  particular  property  of  a  connection  model, 

xr(G),  gives 

ln(P(G))  +  1  +  A0  +  ^  A iXr(G)  =  0.  (14) 

Gee 

1)  The  probability  assignment  for  the  connection  model 

=  (15) 
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2)  The  Hamiltonian  of  a  connection  model  is 


The  definition  of  the  free  energy  is 


H(G)  =  ^\iXi(G).  (16) 

i—1 

3)  The  Partition  Function  of  the  connection  model  is 

Z(G)  =  exp  (1  +  A0)  =  X]  exp (-H(G)).  (17) 

G&S 

4)  The  Interpretation  of  an  instance  of  a  connection  model, 
G,  is  a  sample  drawn  from  an  ensemble  8  of  connection 
models. 

5)  This  definition  is  equivalent  to  that  of  a  physical  system 
defined  over  its  microstate,  provided  the  temperature  is 
finite. 

6)  It  is  always  possible  to  compute  the  expected  value  of 
any  connection  property  of  the  ensemble,  x(G ),  over  the 
particular  model  as 

E[x(G)\  =  Y,P(G)x(G).  (18) 

Ges 

7)  The  way  of  arriving  at  a  definition  of  the  Hamiltonian 
means  we  have  a  way  of  specifying  the  interaction  model 
for  our  network  when  we  have  a  probability  model  for 
the  connections  instead  of  "physical  model". 

8)  We  do  not  need  to  compute  the  statistical  averages  over 
the  connection  probability  distributions  directly,  instead 
we  can  work  from  the  definition  of  the  partition  function 
of  our  connection  model,  which  is  often  easier. 


C.  Some  Observations  Based  Upon  the  Connection  Model 
Partition  Function 

When  one  defines  the  partition  function  Z  (p),  it  is  convient 
to  introduce  p,  a  scaling  parameter  or  the  inverse  temperature 
for  a  physical  system.  For  a  collection  of  data  {/(a;*)}  with  a 
probability  vector,  | p),  the  partition  function  is 

Z<yp)  =  exP  =  exp  (-/u  (p\f{x))) .  (19) 

i 

Take  first  derivative  of  Z(p)  with  respect  to  p 

=  (p\f(x))Z(p).  (20) 


Rearrangement  of  this  expression  gives  the  definition  of  the 
expected  value 

-milzW]=Wix))  =  {,}’  (21) 


which  is  the  definition  of  mean  of  the  data  set  {f(xi}}.  Thus, 
if  one  can  find  the  partition  function  for  a  data  set,  the  mean 
can  be  calculated  from  it. 

A  simple  observation  is  that  the  left  hand  side  of  this 
equation  is  equivalent  to 


1  d  Z(p) 
Z{p)  dp 


d 

dp 


(In  Z). 


(22) 


Ffi,  =  ~  In  (Z(p)) ,  (23) 


SO 

(/) = 

dfi 

Take  the  second  derivative  of  which  gives 

_d_  ( _ 1  dZ(p)\ 

dp  V  Z(p)  dp  ) 

i^)2  ,  1 

z^p)  Z(p)  dp* 
=  -</>2  +  </2) 

=  a/- 


(24) 


(25) 


the  expression  for  variance  is  the  second  derivative  of  the  free 
energy: 

^  =  (f)  -  iff  ■  (26) 

The  variance,  crj,,  can  be  used  to  define  the  heat  capacity  of 
the  network  function  f  is 


Cf  =  p2oj.  (27) 

Note,  there  are  a  variety  of  physics  based  variables  that  can  be 
defined  as  derivatives  of  the  partition  function  including  pres¬ 
sure,  average  energy,  capacity,  temperature,  entropy,  etc.  These 
statistical  mechanics  variables  may  be  worth  considering  in  a 
broader  context. 

Any  characteristic,  C,  of  a  connection  model  can  find 
expected  value  from  the  formula 

(G)  =  Z~\G)  Y,  C{G)  exp  (-if  ((G))) .  (28) 

G 

To  measure  the  parameter  G,  we  simply  adjoin  it  as  a  term 
added  to  the  Hamiltonian 

n 

H{G)  =  Y  PiVi(G)  +  0C.  (29) 

i—l 


Calculating  the  expected  value,  (G),  for  a  linear  interaction 
potential  from  the  free  energy  as 


(G) 


dF 

~dd 


9=0 


(30) 


this  generalizes  to  enable  us  to  calculate  correlation  functions 
of  the  connection  model  as 


(vivj...vi)  =  Z 


-i 


d  d 


d 


Z. 


(31) 


[dp,  dpj"'dpl \ 

We  can  then  calculate  the  correlation  between  two  elements 
as 

W  jp 

(32) 


d2F 

dpidp 


=  (z'iZ'j)  -  Wi)  (Vj)  , 


while  the  extension  to  higher  orders  is  straight  forward.  Thus, 
instead  of  interpreting  A iXi(G)  as  the  Lagrange  undetermined 
multiplier  times  some  property  of  the  connection  model,  it  can 
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be  interpreted  as  an  interaction  potential  between  source,  i, 
and  source  j.  The  A,  can  be  interpreted  as  scaling  parameter 
p,  a  field  coupling  parameter,  or  the  inverse  temperature.  It 
is  possible  to  posit  a  variety  of  interaction  models,  work  out 
their  consequences  and  work  out  the  equivalent  probability 
distributions  for  them  as  well. 

III.  Synthesisizing  An  Interaction  Model  as  a 
Solution 

The  more  we  know,  the  more  complicated  the  form  of  the 
probability  assignment  becomes  up  to  a  point.  In  creating  a 
network  that  fuses  weighted  information  from  different  sources 
we  are  creating  the  equivalent  of  a  physical  interaction  system 
that  has  an  underlying  physics  that  we  can  strive  to  understand 
and  exploit  this  knowledge.  Thus,  an  algorithm  when  looked  at 
this  way  is  a  physics  model  based  on  interpreting  our  weighing 
of  the  data  from  different  sources  as  an  underlying  interaction 
model.  Any  assignment  of  probabilities  can  always  be  viewed 
as  the  solution  to  a  Bayesian  Maximum  Entropy  problem. 
When  this  information  is  combined  in  a  state  vector,  we  have 
the  possibility  of  hidden  laws  emerging.  Given  that  is  the  case, 
then  we  can  ask  the  question:  "What  is  the  Maximum  Entropy 
problem  that  an  assignment  that  a  weighting  scheme  is  the 
answer  to?"  For  other  examples  not  related  to  this  question, 
see  the  suggestive  paper  by  Kesavan  [8], 


The  partition  function  (PF)  is 

Z,{G)  =  y>xp(-iJ(G)) 

G 

=  ^]exp  (-n^2aij(G)) 

CLij  i  <  j 

1 

=  n  exp (-/ztzy(G)) 

i  <  j  aij—  0 

= 

i  <  j 

=  (1  +  exp  (-^))W  . 

The  Model  1  free  energy  is  given  by 


(35) 


^Modell)  =_ln  (z«(G)) 


In  (1  +  exp  (-/z)) , 
(36) 


while  the  mean  is 


,  (Model  1)  _  iFf 


G) 


j  p\  ^ivioaei  i)  _  ft  _ 

dfi  (1  +  exp  (/i)) ' 


The  standard  deviation  is 

2 


o$  (Model  1)  =  — 


(2)  exp  M 
(1  +  exp  (/z)) 


(37) 


(38) 


A.  Partition  Functions,  the  Free  Energy’,  and  Models  for  iT (G) 

Graph  theory  has  been  used  as  a  model  for  social  networks 
since  the  eighties.  The  idea  of  applying  statistical  mechanics 
dates  back  to  the  seminal  paper  by  Frank  [3]  and  Strauss 
[14],  while  a  recent  overview  is  found  in  [13].  Park  [10] 
has  developed  a  detailed  statiscal  mechanics  study  of  graph 
models.  This  work  is  adpoted  from  these  three  sources.  We 
have  introduced  the  notion  of  connection  models  rather  than  a 
graph  model  because  the  notion  of  a  connection  model  is  more 
evocative  than  a  graph.  Also,  it  is  consitent  with  the  older  body 
of  work  reflected  in  Ross  on  the  usage  of  probability  models 
in  the  study  of  general  probability  models  [11]. 

The  simplest  form  for  the  Hamiltonian  for  a  connection 
model  is  a  model  where  we  know  the  expected  number  of 
interactions  for  the  model,  namely:  (m),  so  H(G)  becomes 

H^(G)  =  /zm(G).  (Model  0) 

This  model  is  trivial,  so  the  next  simplest  Model  is  when 
adjacency  matrix  A  (G)  has  components  a. ij 

_  J  1  if  source  i  is  connected  to  source  j 
13  1  0  if  source  i  is  not  connected  to  source  j 

(33) 

The  number  of  interactions,  m,  is  given  by 

m(G)  =  ^  ay  (G) ,  (34) 

i  <  j 

while  the  Hamiltonian  is 


and  the  heat  capacity  is 


G 


(Model  1) 
/ 


=  “M 


(2)  exP  (M) 


(39) 


(1  +  exp  (/lz)) 

To  compare  the  interaction  model  to  the  underlying  connec¬ 
tion  interaction  probability  model,  we  reparameterize  p  as 

1 


V  = 


so 


and  therefore 


exp  (/z)  = 


(1  +  exp  (n)) 
1  ~P 


P  = 


P 

exp  (— /z) 

(1  +  exp  (-/z)) 


Thus,  solving  for  (1  —  p), 

P  exp  (/z)  =  (1  ~p)  = 

The  first  moment  is 

</>(1)  = 

while  the  standard  deviation  is 

2 


(1  +exp(-/z))' 


P, 


O' 


The  heat  capacity  is  given  by 


(40) 

(41) 

(42) 

(43) 

(44) 


(!)  =  (2)  (P  ~  P2)-  (45) 


iT(G)  =  p  ay  (G) .  (Hamiltonian  Model  1) 

i  <  j 
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The  partition  function  is 

Z^G)  =  exp  (— Fm)  =  (1  +  exp  (-p))^ 

=  (l-p)(S). 


(47) 


Therefore  the  probability,  P(G)  for  the  connection  model 
ensemble  is 


P(G)  = 


exp  (-H) 
Z 

1  ~P 
P 


exp(— pm) 

(1  +  exp(-/r))(2 
(1  —  p)(a) 


=  pm(l-p)^hm.  (48) 

This  is  the  probability  for  a  graph  for  which  each  of  the  (”) 
possible  edges  appear  with  probability  p,  which  is  the  well 
known  Bernoulli  random  graph  model  studied  by  Erdos  and 
Renyi. 

B.  General  Model 

A  more  general  model  is  to  replace  pi  with  p^  so  the 
Hamiltonian  is 


H(G)  =  I'rd'rMD. 

i  <  j 

then  the  partition  function 

z(Model3)(G)=  JJ  + 
i  <  j 

The  free  energy  is 

^Model  3)  _  —  ln  ^(Model  3) 

=  —  In  (l  +  e-^v] 


(General  Model) 


(49) 


i  <  j 


therefore  the  first  moment  is 


^  j ^  (Model  3)  d-Fii 


(Model  3) 


dp 


(1 


O’ 


the  standard  deviation  is 

,/2  p(Model  3) 

<j2f  (Model  3)  =  M 


e^'3 


dp? 


and  the  heat  capacity  is 


(Model  3) 


(1  +  eMo)' 
e(^o) 


^*3/  I  \  \  2  ' 

(1  +  eK) 


Then  we  can  reparameterize  p  as 


1 


Pij  = 


(50) 


(51) 


(52) 


(53) 


(54) 


(1  +  eMii) ' 

Note,  some  simplified  models  beyond  Model  1  are  special 
cases  with  pij  =  (/y  +  Pj)  for  Model  2  and  /i.(J  =  p  for 
Model  1. 

In  general,  one  can  specify  interaction  Hamiltonian  as 

H  =  A ijUij, 

i  <  j 


(General  Hamiltonian  Model) 


where  A ^  is  parameter  that  couples  each  source  together  and 
is  the  connection  sequence  satisfies  z/;  =  ■  cry.  The  partition 

function  is  _ 

Z  =  Yl(l  +  e~*«),  (55) 

i<j 

while  the  free  energy  is 

F  =  -  ln(l  +  e_A«)  .  (56) 

i  <  j 

Note  the  probability  of  an  interaction  between  source  i  and  j 
is  the  expected  number  of  the  connection  sequence,  then 

dF  1 

Pij  -  Wij)  =  dXd.  =  (i  +  eA „)■ 

Underlying  basis  for  the  parameterization  that  gives  us 
Bernoulli  model  with 

P{G)=p?j{  1-Pijph™.  (58) 

Note,  the  term  CiOi  can  be  added  to  Hamiltonian  without 
changing  the  probability  model  we  have  derived.  This  enables 
us  to  relate  ( Ci )  to  characteristic  of  the  interaction  model. 
Furthemore,  the  Bernoulli  probability  model  derived  from 
different  types  of  interaction  are  all  the  same.  So,  one  can 
conclude  that  any  interaction  model  that  can  be  cast  into  the 
above  form  is  a  hidden  Markov  model.  This  observation  allows 
us  to  connect  many  seemingly  disparate  facts. 

C.  Example:  Decision  Fusion 

A  design  is  created  from  disparate  elements,  that  when 
combined  form  a  composition  (true  of  artistic  or  engineering 
design).  Each  discipline  has  "atoms  of  composition"  or  atomic 
units  that  are  fused  together  to  form  a  specific  instance  of  a 
design.  In  this  instance,  the  rules  for  assigning  connections 
are  the  rules  of  composition.  In  artistic  designs,  the  method 
of  fusion  that  constitute  the  finished  product  is  based  on  the 
individual  interpretations  of  the  group  aesthetic  that  falls  under 
the  aegis  of  "artistic  temperament".  For  engineers,  the  group 
aesthetic  must  have  some  familiarity  with  the  requirements 
of  "physical  reality".  While  it  is  difficult  to  arrive  at  a 
mathematical  criteria  that  helps  clarify  the  direct  problem  of 
the  process  of  composition,  the  inverse  problem  of  determining 
how  the  composition  was  generated  is  possible  because  we  can 
infer  law  based  on  the  probability  assignment. 

Our  goal  in  this  paper  is  to  outline  the  mathematics  of  com¬ 
position  of  decisions,  which  we  have  already  done.  Decision 
Composition  can  be  viewed  as  a  fusion  of  individual  atoms 
of  composition  which  are  weighted  components  with  these 
properties: 

1)  The  numerical  weights  of  the  atoms  of  composition  of 
a  design  can  be  assigned  values  between  zero  and  one. 

2)  The  sum  of  the  weights  can  be  normalized,  so  the  sum 
of  the  weights  of  all  the  atoms  of  composition  in  an 
instance  of  a  design  add  up  to  be  one. 

3)  These  two  properties  are  mathematically  equivalent  to 
an  assignment  of  probabilities,  so  they  can  be  treated  as 
probabilities. 
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For  example,  a  particular  design  can  always  be  viewed  as 
an  instance  of,  or  a  particular  sample  drawn  from  a  space  of 
possible  compositions  that  have  common  elements.  Proof  of 
a  mathematics  theorem  can  also  be  viewed  as  a  design  with 
elements  drawn  from  axiomatic  systems  combined  with  the 
glue  of  the  rules  of  inference.  Thus  conceptually  it  is  possible, 
in  principle,  to  consider  a  proof  as  a  composition  relative  to 
some  system  of  music  or  as  a  painting.  From  the  Bayesian 
perspective,  any  sufficiently  complex  problem  has  incomplete 
knowledge  for  us  to  try  and  reason  through. 

Reasoning  from  the  perspective  of  Lack  of  Knowledge  is 
equivalent  to  probabilistic  reasoning.  Laplace,  Bayes,  Gibbs, 
Shannon,  Jaynes  are  source  of  this  idea  [5], [6],  [7],  [12]. 
It  has  proven  to  be  the  key  to  creating  a  real  Information 
Age.  " Probability  is  the  logic  of  reasoning  in  the  presence  of 
uncertainty.”  Assignment  of  probabilities  based  on  available 
information  is  Bayesian  reasoning.  This  provides  a  solution 
of  assignment  of  probabilities  which  is  a  Maximum  Entropy 
Problem  (MEP).  Thus  a  decomposition  of  a  design  into 
probabilities  can  be  viewed  as  a  ( there  is  no  the )  solution 
to  a  MEP.  Given  that  MEP  is  the  method  that  provides  the 
answer,  then  we  can  ask  the  question:  "  What  is  the  Maximum 
Entropy  problem  that  an  assignment  of  design  weights  is  the 
composition  an  answer  to?”  Being  able  to  answer  this  question 
for  "design  classes"  provides  us  a  means  to  classify  Design  as 
a  mathematical  problem.  Design  algorithms  can  be  then  be 
ordered  in  terms  of  the  complexity  of  the  problem  solved. 
By  providing  the  mathematical  basis  for  synthesizing  design 
as  a  composition,  we  hope  to  provide  a  pathway  to  analyze 
decisions  from  the  perspective  of  design.  This  allows  decisions 
to  be  understood  on  a  more  quantities  basis  and  ultimately 
bring  the  decision  maker  in  the  fusion  process  in  a  more 
quantitative  process.  This  is  illustrated  with  a  simple  example. 

There  are  three  ways  to  think  about  how  decisions  are 
synthesized:  a  branching  process,  a  fusion  problem  from 
multiple  information  sources,  or  a  grey  scale  picture  where 
the  lighting  reflects  the  degree  of  connection  between  the 
individual  components.  Each  of  these  three  methods  is  useful 
depending  on  the  problem  domain. 

The  visual  model  is  a  way  of  reconstructing  a  complicated 
process  where  there  are  no  particular  analytical  details  known 
for  its  construction.  By  assigning  different  weighting  schemes 
we  can  see  a  decision  emerges  out  of  a  morass  of  disparate 
data. 

For  a  branching  process,  the  starting  point  can  be  thought  of 
as  the  final  decision,  and  the  branches  constitute  the  elements 
that  went  into  the  decision.  At  each  of  the  branching  levels  the 
decisions  have  weights  that  must  normalize  like  probabilities, 
so  the  formulation  of  the  connection  model  adjacency  matrix, 
A,  reveals  structure  associated  with  the  retrodiction  process. 
The  structure  of  the  operator  algebra  gives  an  algebraic  un¬ 
derstanding  of  decisions  in  terms  of  "operator  logic"  of  the 
matrix. 

For  fusion  problems  associated  with  target  tracking,  the 
third  method  is  most  useful.  The  asynchronous  multi-sensor 
track  fusion  problem  can  be  stated  as  follows:  Given  a  number 


of  asynchronous  valid  tracks,  {Xfitki  \tki),Pi(tki  \tki)},  i  = 
1,2,  that  arrive  at  the  fusion  center  during  the  time 
interval  [tk-i,tk\,  find  the  best  estimate  in  the  minimum 
mean  square  sense  of  the  system  state  at  time,  tk ,  when  it 
is  computed  according  to  the  fusion  rule. 

n 

Xf(tk\tk )  =^^/Li(tki)Xi(tki\tki)  (59) 

*=i 

where  the  Li ’s  are  unknown  weighting  matrices  to  be  deter¬ 
mined.  The  two  constraints  imposed  on  the  fused  track  are 
unbiased  if 

n 

=  I-  (60) 

i=l 

This  represents  the  first  constraint  on  the  choice  of  the 
weighting  matrices.  If  we  solve  for  Ln,  we  have 

n—  1 

Ln  =  ^  Li$(tki,tkn)-  (61) 

i=l 

The  error  covariance  matrix  of  the  fused  track  can  be 
defined  as 

Pf{tk\tk)  =  E{Xf{tk\tk)Xf{tk\tk)  }  (62) 

which  allows  determination  of  the  weights  L,;  which  define 
the  optimal  asynchronous  track  fusion  filter.  Further  details 
about  this  track  fusion  approach  are  found  in  [2]  and  [1].  The 
solution  is  encapsulated  in  theorems  like  this: 

Theorem  1:  The  error  covariance  matrix  of  the  fused  track 
using  the  fusion  rule  is  given  by 

n— In— 1  n— 1  n— 1 

Pf(k\k)  =  'y  '  y  '  LiMijL'j  +  y  '  LiNi  +  y  '  IVjZ^  +  Mn. 

i= 1  7=1  i=  1  i=  1 

(63) 

Theorem  2  :  The  minimum  mean  square  solution  of  the 
asynchronous  track  fusion  problem  using  the  fusion  rule  is 
given  by 

n 

Xf(tk\tk)  =  LjXi(tki\tki),  (64) 

i—i 

Pf(tk\tk )  =  LML  +LN  +  NL  +  Mn,  (65) 
where 

[Li  L2  ...  L„_i]  =  -NM-\  (66) 

n—  1 

Ln  =  $(4,4„)  -  ^2  Li${tki,tkn).  (67) 

i- 1 

The  reason  this  general  example  has  been  presented  is  that 
it  is  indicative  of  all  fusion  algorithms;  they  are  variations 
on  a  similar  theme.  With  the  proper  redefinitions  of  matrices, 
both  track  and  data  fusion  amount  to  the  same  thing:  weighted 
(positive  semi-definite  and  the  sum  is  normalized  to  one) 
combinations  of  data.  The  results  can  be  extremely  com¬ 
plicated,  difficult  to  understand,  difficult  to  predict  in  terms 
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of  performance,  and  difficult  to  determine  the  algorithm’s 
underlying  correctness,  but  they  have  the  same  underlying 
mathematical  form  when  understood  properly.  Given  this 
observation,  an  alternative  suggests  itself  as  a  means  of  un¬ 
derstanding  these  types  algorithms  based  on  maximum  entropy 
analysis.  An  example  is  the  Bar-Shalom-Campo  Fusion  Rule: 


[12]  Shannon,  C.  E.  and  Weaver,  W.,  The  Mathematical  Theory  of  Communi¬ 
cation ,  University  of  Illinois  Press,  Urbana,  1964.  (Originally  published 
in  1949,  this  contains  a  reprint  of  Shannon’s  1948  papers  that  appeared 
in  the  Bell  System  Technical  Journal.) 

[13]  Skvoretz,  J.,  " Complexity  Theory  and  Models  for  Social  Networks ", 
Complexity ,  Vol.  8,  No.  1,  2003. 

[14]  Strauss,  D.,  "On  a  General  Class  of  Models  for  Interaction ",  SIAM 
Review ,  Vol.  28,  No.  4,  1986. 


Xf(k\k)  =  X2(k\k)  +  (P22  -  P21)U~1[X1(k\k)  -  x2(k\k)}, 

(68) 

Pf{k\k)  =  P22  -  (P22  -  P2l)U~l[P22  -  Pi2].  (69) 

This  result  is  simple  enough  to  work  out  the  equivalent 
interaction  model  for  and  will  be  the  subject  of  future  work. 

IV.  Concluding  Comments 

The  translation:  Interaction  Model  <^>  Probability  Model 

provides  a  richer  interpretation  of  "physics".  This  allows 
us  to  consider  more  general  types  of  interactions  that  the 
statistical  mechanics  crank  can  be  brought  to  bear.  Fusion 
of  weighted  information  from  different  "atomic  composition" 
elements  is  equivalent  to  creating  an  interaction  system  with 
an  underlying  physics  that  we  can  strive  to  understand.  There 
is  always  a  design  physics  model  based  on  interpreting  the 
weights  associated  with  the  components  of  a  composition. 
This  provides  an  entirely  new  method  to  analyze  Design 
mathematically  from  the  perspective  of  inverse  problems  based 
on  the  MEP  perspective.  It  also  provides  a  new  basis  for 
the  mathematization  of  design.  This  is  an  area  that  requires 
further  development.  The  field  has  rich  predictive  capability, 
the  question  is  how  useful  it  is  when  applied  to  specific 
problem  domains.  Each  field  raises  interesting  problems,  and 
even  more  interesting  problems  will  be  raised  as  the  initial 
ones  are  solved. 
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